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METHOD OF AND APPARATUS FOR IMPROVING PRODUCTIVITY 
OF HUMAN REVIEWERS OF AUTOMATICALLY TRANSCRIBED 
DOCUMENTS GENERATED BY MEDIA CONVERSION SYSTEMS 

FIELD OF THE INVENTION; 

The present invention relates to the field of media conversion systems. More 
particularly, the present invention relates to an apparatus for and method of 
improving productivity of human reviewers of transcribed files generated by media 
conversion systems. 

BACKGROUND OF THE INVENTION: 

Recent advances in media conversion technology, such as speech-to-text 
conversion and optical character recognition (OCR), have made it possible for 
computers to perform tasks such as transcribing messages or documents dictated 
by a user. Dictation systems for personal computers are now fairly common. Also 
available are scanning or OCR systems that are capable of converting the content 
of a printed document into computer readable form. These systems, however, are 
sometimes inaccurate. The users often need to proof-read and make corrections to 
the output of these systems. 

Also emerging as a new technology is the unified messaging system. The 
Unified Messenger™, manufactured by Octel Communications Corporation of 
Milpitas, California, is one such system. These unified messaging systems 
integrate a user's fax messages, voice messages and e-mails in a single logical 
mailbox such that they can be accessed by telephone or computer. For instance, 
the Unified Messenger can convert an e-mail message into a voice message so that 
it can be read to the user over the telephone. Such a system is described in U.S. 
Patent 5,557,659, which is incorporated herein by reference. 

Data entry personnel are currently utilized for the conversion of an audio 
message to a text message sent to a user's paging device. In such a system, a 
caller desiring to send a message, telephones the user's paging service and 
communicates the message to a data entry employee of the paging service. This 
employee enters the message into a computer and then transmits it to the user's 
paging device. The text message entered by the employee of the paging service is 
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then displayed on the display of the user's paging device. The use of human 
employees in the text entry and transcription of audio messages is expensive and 
inefficient. Current automated systems are not reliable and accurate enough to be 
used for a fully automated messaging or transcription system. Accordingly, what is 
needed is an automated system which also employs human reviewers to maximize 
the efficiency and effectiveness of the reviewers. 

When documents are dictated and recorded as an audio file, a person 
transcribing the document plays the audio file and enters text representing the 
document recorded on the audio tape. The use of human employees in text entry 
and transcription of audio messages is expensive and inefficient. Automated 
speech-to-text conversion systems are also currently available to convert audio 
recordings into a text document. However, such conversion systems are 
inaccurate, requiring the users to proof-read and make corrections to the output of 
the systems. 

SUMMARY OF THE INVENTION: 

An apparatus for improving productivity of human reviewers of transcribed 
documents generated by media conversion systems includes a server/client network 
of computers, memories and file systems. The server receives and stores voice 
files created by users of the system. The server is configured for coupling to a 
speech-to-text media conversion system to receive converted text files of the audio 
voice files. The server analyzes the converted text files and routes the converted 
files to the appropriate reviewers according to an adaptive algorithm. The 
converted files are displayed on the assigned reviewer's screen at the reviewer's 
workstation. To aid the reviewer in pinpointing potential errors, the workstation 
displays different segments of the converted files in different colors to reflect 
different confidence levels of transcription accuracy. Portions of the original voice 
message that correspond to the potential errors are played back for the reviewer. 
The reviewers 1 workstations also perform productivity enhancing functions such as 
spelling and grammar checking. After the reviewer has made all the necessary 
corrections, the reviewed files are transmitted back to the server to be stored and 
accessed by the users. A user database in the server is also updated to store 
recurrent user-specific errors corrected by the reviewer. A language analysis 
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system is also disposed to adaptively correct user-specific errors in future reviews 
according to the information in the user database. 

BRIEF DESCRIPTION OF THE DRAWINGS: 

Figure 1 illustrates in block diagram form a system of the preferred 
embodiment of the present invention, including a server, a media conversion 
system, and a network coupling reviewer workstations, communication devices, 
and output peripherals to the server. 

Figure 2 is a flow diagram illustrating the operations of the preferred 
embodiment of the present invention when a reviewer logs on. 

Figure 3 is a flow diagram illustrating the operations of the server in selecting 
an appropriate reviewer. 

Figure 4 is a flow diagram illustrating the operations of the present invention 
after a reviewer is selected. 

Figure 5 is a flow diagram illustrating the operations of the present invention 
after the text document is reviewed. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT: 

Figure 1 illustrates in block diagram form a system of the preferred 
embodiment of the present invention. The present invention is directed to an 
apparatus for and method of improving productivity of human reviewers of 
automatically transcribed documents generated by media conversion systems. It is 
the purpose of this patent to cover all media conversion systems including but not 
limited to speech-to-text, text-to-speech, OCR and hand-writing recognition 
systems, as well as other information processing systems which may require 
human supervision. However, for the sake of simplicity, the exemplary system 
illustrated in Figure 1 and described herein is configured for coupling to only a 
speech-to-text media conversion system. However, as should be apparent to those 
skilled in the art, the preferred embodiment of the present invention is easily 
adapted for use with other types of media conversion systems or information 
processing systems. 

As shown in Figure 1 , a speech-to-text media conversion system 1 25 is 
coupled to a server 100. The speech-to-text media conversion system 125 
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preferably includes a conventional computer having a voice processing card such as 
a SoundBlaster 16 from Creative Labs, and running voice recognition software, 
such as Naturally Speaking from Dragon Systems, Inc. Alternately, the speech-to- 
text system 125 is integral to the server 100. The speech-to-text media conversion 
5 system 1 25 automatically converts audio voice files to a text document. 

The server 100 is preferably coupled through a telephone switching network 
1 10 to a plurality of reviewer workstations 120 and a plurality of input devices. 
The input devices include any appropriately configured device through which a user 
can communicate a message to the server 100. In the exemplary system, 
10 illustrated in Figure 1, a telephone set 130, and a dictation machine 131 are 
f j included as input devices. Alternately, the input devices are coupled to the server 
O 1 00 through a PBX or other appropriate network. In a further alternate 

embodiment, the server 100 is coupled to the reviewer workstation 120 via a 
o computer network such as the Internet. The server 100 is also coupled to a 
1 Etif plurality of output peripherals including a fax machine 1 33 and a printer 1 32. The 
U reviewer workstations 1 20 can be located in a centralized location or can be 
^ located at the reviewer's office or home and connected to the server 100 through 
^ the telephone network by a modem. 

H Preferably, the server 100 includes an integral language analysis system 101 

2C|X for improving the accuracy of the text document generated by the speech-to-text 
O media conversion system 125. It is important to note that the language analysis 
system 101 performs functions different from those of the speech-to-text media 
conversion system 125. While the speech-to-text media conversion system 125 
analyzes and translates the voice files phonetically, the language analysis system 
25 101 , such as the Mikrokosmos semantic analyzer from New Mexico State 

University at Las Cruces, New Mexico, analyzes the context of the resulting text 
document and corrects phonetically correct but contextually inappropriate words. 
For example, if the speech-to-text media conversion system 1 25 converts the 
sentence: "I red a book", the language analysis system 101 will substitute "read" 
30 with "red" in order to correct the resulting text document. In addition, user- 
specific mistakes that have been corrected in previous reviews of documents 
submitted by the user are also stored in a user database 102 within the server 100. 
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Preferably, the language analysis system 101 is disposed to adaptively process the 
text file according to the information stored in the user database 102. 

The user database 102 is also integral to the server 100 and contains 
information pertaining to the identities of the users. In addition, the user database 
102 contains a list of reviewers who have previously reviewed documents 
submitted by the user. Furthermore, the user database 102 includes a list of 
preferred reviewers designated by the user. The preferred reviewers are given a 
higher priority during the assignment process, after the user has submitted a new 
document. 

In the preferred embodiment of the present invention, the server 100 also 
includes a reviewer database 104. The reviewer database 104 contains a plurality 
of skill sets for each of the reviewers. A skill set is a list of keywords that a 
reviewer is familiar with. For instance, for a reviewer proficient in reviewing or 
transcribing medical documents, his skill set will include common medical terms. 
The reviewer database 104 is updated when a new reviewer is hired, or when a 
reviewer acquires a new skill set. The reviewer database 104 also contains other 
information, such as the number of reviews made and the average time a reviewer 
spends to finish reviewing a document. Other information useful for evaluating the 
performance of the reviewer is also stored in the reviewer database 104. 

The server 100 also maintains a reviewer log 106 which comprises a list of 
currently available reviewers. The reviewer log 106 is updated whenever a 
reviewer logs on or off the server 100, or when a reviewer is assigned a new 
reviewer package. The server 100 uses this information as a basis for determining 
whether a reviewer is available to accept a newly submitted document. 

The server 100 includes the control processor 108 which is coupled to the 
language analysis system 101, the user database 102, the reviewer database 104 
and the reviewer log 106. The control processor 108 controls the acceptance of a 
new document from a user and the assignment of the document to a reviewer. The 
control processor 1 08 is also coupled to communicate with the network 1 1 0 in 
order to send a document to a reviewer's workstation and to communicate 
instructions to and receive confirmations from the reviewer. 

The server 100 also includes a storage device 103 for storing documents to 
be sent to or received from reviewers. The storage device 103 is preferably a hard 
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disk drive. Alternately, the storage device 103 is any appropriately configure media 
storage device. 

After a voice file is created, it is transmitted to the speech-to-text media 
conversion system 1 25 for conversion into a corresponding text file, and then to 
the language analysis system 101 . The language analysis system 101 checks the 
text document for contextually and grammatically incorrect words as described 
above. The server 100 then determines which one of the currently available 
reviewers listed in the reviewer log 106 should review the text document. The 
server 100 makes this decision by first extracting a list of keywords from the text 
file. The keywords are extracted from the text file by searching the document and 
comparing it to a keyword database stored in the storage device 103. The server 
1 00 then matches the list of keywords to a skill set within the reviewer database. 
If the skill set of a reviewer matches the keywords, and if that particular reviewer is 
available, the server will route the text document file and pointers corresponding to 
the voice file to the workstation 1 20 associated with the reviewer. Alternately, the 
actual voice file is also routed to the reviewer's workstation 1 20. By assigning text 
files to reviewers with specialized knowledge and skills in a particular field, the 
productivity of the reviewers is increased. 

To fully maximize the throughput of the reviewers, each of the reviewers' 
workstations 120 are equipped with a color monitor 122 and a microphone 124. 
Preferably, the reviewers' workstations 120 also include other conventional input 
devices such as a keyboard, mouse and a foot pedal. The color monitor 122 is 
preferred because it is desirable to display the transcribed text file in different 
colors. In the preferred embodiment, sections of the transcribed text file having a 
low confidence level are displayed in red, sections of the text having a high 
confidence level are displayed in green and sections of the text having a medium 
confidence level are displayed in yellow. Alternately, any desired color combination 
can be used to highlight the different confidence levels within the text document. 
With the contrasting colors representing the different confidence levels, the 
reviewer is able to quickly pinpoint potential errors without carefully reading every 
single word in the document. Alternately, a monochrome monitor is included at the 
reviewer's' workstation 1 20. In this alternate embodiment, sections of the 
transcribed text file having a low confidence level are highlighted and sections of 
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text having a high confidence level are displayed normally. In a further alternate 
embodiment, sections of the text file having a low confidence level are underlined, 
and sections of the text file having a high confidence level are italicized. 

The microphone 1 24 further increases productivity by obviating the need to 
use a mouse or other conventional input device to navigate around the document. 
In the preferred embodiment, the workstation 1 20 is voice activated and controlled 
by simple voice commands such as "select", "delete", "scroll down", etc. 
Alternately, the workstation 1 20 utilizes the keyboard to enter any changes to the 
text file, and the mouse to select a portion of the text document and the foot pedal 
to start and stop the replay of the recorded audio voice file. 

Figure 2 is a flow diagram illustrating the operations of the server in selecting 
an appropriate reviewer. As shown at the step 200, a user first calls the system 
through an input device such as the telephone 130 or the dictation machine 131 
and records a voice file. The voice file is then transferred to a speech-to-text media 
conversion system 125 where it is automatically transcribed into a corresponding 
text file at the step 202. The voice file is then stored in conjunction with the text 
file in the memory storage device 103. At the step 204, the text file is analyzed by 
the language analysis system and marked with confidence levels indicating the 
possibility of errors as described above. The language analysis system 101 
determines which parts of the text document are likely to be erroneous, and which 
parts are likely to be correct. The text file is also scanned by the language analysis 
system 101 at the step 206 to produce a list of keywords corresponding to the 
text file. At the step 208, the list of keywords corresponding to the text file is 
compared with the skill sets in the reviewer database 104. The reviewer database 
1 04 contains a list of keywords relating to the field with which the reviewer is 
familiar. By matching the words contained in the skill sets with the words in the 
list generated by the speech-to-text media conversion system 125, the reviewers 
whose skill set has a higher correlation with the list of keywords will be accorded a 
higher priority. Then, the reviewer log 106 is consulted to determine the current 
availability of the reviewers at the step 210. The available reviewer with the 
highest priority is selected at the step 212 and a reviewer package containing the 
voice file and the text file, as well as the criteria for selection is sent to the 
workstation 1 20 of the selected reviewer at the step 214. Within the preferred 
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embodiment of the present invention, the assignment process can be manually 
overridden such that a user may request a particular reviewer to review his 
document. 

In some applications, it is desirable to give a higher priority to a reviewer who 
5 has prior experience with the user. In these applications, the system will store a 
number representative of the number of times a reviewer has reviewed documents 
from a particular user. This feature can be used in conjunction with the features 
described above to select the most appropriate reviewer and to maximize the 
reviewer's review of documents with which he has a familiarity. 
10 Figure 3 is a flow diagram illustrating the operations of the preferred 

O embodiment of the present invention when a reviewer logs on to the server 100. 

As shown, a reviewer logs on to the server 100 through a workstation 1 20 by 
In entering his name and an identity verifying a password at the step 300. The 
w reviewer is then prompted to download a new reviewer package from the server 
1 % 100 at the step 302. If the reviewer desires to receive a new assignment, the 
i t% reviewer enters the appropriate commands to download the appropriate files from 
JL the server 100 to the reviewer's workstation 120. After a reviewer package is 
i, i downloaded, the reviewer's workstation 1 20 displays the criteria used for selecting 
the reviewer at the step 304. The criteria for selecting the reviewer is displayed so 
2Qh that the reviewer is able to quickly determine if the system has correctly assigned 
O tasks matching his skill set. The reviewer may choose to review the transcription 
or to reject the assigned task at the step 306. If the task is rejected, the reviewer 
package is deleted, the server selects a second reviewer and a new reviewer 
package is transmitted to the workstation 1 20 corresponding to the second 
25 reviewer at the step 308. Once a reviewer accepts the new assignment, the 
review of the text file is performed at the step 310, as illustrated in the steps 
included in Figures 4 and 5. 

Figure 4 illustrates the operations of the present invention after a reviewer is 
selected. After a reviewer chooses to accept the assignment at the step 400, the 
30 text file is displayed in different colors according to the different confidence levels 
at the step 402, so that the reviewer can quickly pinpoint problematic or low 
confidence level areas, as described above. The reviewer then selects any 
appropriate portion of the text within the message using voice activated commands 

- 8 - 
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or other conventional input devices at the step 404 during their review. When a 
section of the text file is selected, preferably a corresponding section of the audio 
voice file is played for the reviewer at the step 406. The reviewer has the ability 
through the client computer 1 20 to control the speed of this audio replay. 
Alternatively, the reviewer particularly selects portions of the text for which the 
corresponding audio voice file is played. After listening to the voice file, the 
reviewer then corrects any inaccuracies at the step 408. In the event that the 
reviewer cannot determine accurately what was recorded from the voice file 
because of poor recording quality, the reviewer may establish a connection with the 
user who submitted the document at the step 410. If the author of the message 
also has access to a workstation 1 20, the text file is simultaneously displayed on 
the author's terminal at the step 412. The reviewer then receives comments from 
the author and makes further corrections to the file at the step 414. 

The reviewer then repeats this process and corrects all the inaccuracies 
within the text file at the step 41 6. After the reviewer is finished, a check for 
spelling and grammar mistakes is automatically performed at the step 418, to 
ensure that the document contains a minimal level of errors. 

Figure 5 illustrates the operation of the preferred embodiment of the present 
invention after the review of the text document is complete and all apparent 
inaccuracies have been corrected. As shown in Figure 5, after the reviewer has 
finished their review of the document, the edited text document is transmitted to 
the server 100 to be saved, at the step 500. The edited text document is saved 
within the media storage device 103. The server 100 also saves information 
pertaining to the corrections made by the author in the user database at the step 
502. This information is used in transcriptions of future documents by that author. 
Then at the step 504, the list of available reviewers is updated to add the reviewer 
who finished the last review to the list of reviewers available for new assignments. 
The reviewer database is also updated at the step 506 to store the time necessary 
to review and correct the last file. Finally, at the step 508, the stored text file is 
printed out and sent to the author. The text file may also be sent electronically to 
an intended recipient as an e-mail document. 

An apparatus for and method of improving the productivity of human 
reviewers of computer-processed information has herein been described. This 
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apparatus and method improves the efficiency of human reviewers and improves 
accuracy of an automated conversion system. The present invention also enables 
an integrated messaging system to provide accurate transcriptions of many types of 
documents including converting voice messages into fax messages or e-mail 
messages at a low cost. Furthermore, employing the present invention, an 
integrated messaging system can accurately convert a fax message into a voice 
message to be retrieved by telephone. 

Although the preferred embodiment of the invention uses a speech-to-text 
system for its implementation, it should be apparent to persons skilled in the art 
that other information processing functions, such as text-to-speech, OCR or hand- 
writing recognition functions, can also be implemented within the system of the 
present invention based on the principles herein discussed. Further, there may 
evolve new message types that would allow similar implementations. 
Consequently, this disclosure is not meant to narrow the invention by illustrating 
the use of the speech-to-text media conversion system as an example. It will be 
apparent to those skilled in the art that modifications may be made in the 
embodiment chosen for illustration without departing from the spirit and scope of 
the invention. Specifically, it will be apparent to one of ordinary skill in the art that 
the method of the present invention could be implemented in many different ways 
and the apparatus disclosed above is only illustrative of the preferred embodiment 
of the present invention. 
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CLAIMS 

11. An apparatus for improving productivity of human review of an 

2 automatically transcribed output generated by an information processing system, 

3 wherein the output is generated in response to an input, the apparatus comprising: 

4 a. means for extracting an attribute from the output; and 

5 b. means for selecting one of a plurality of human reviewers based on the 

6 attribute. 

1 2. The apparatus according to claim 1 wherein the attribute represents a 

2i subject matter of the output, and further wherein the human reviewer is selected 

3^ based on a proficiency in the subject matter of the human reviewer. 

tij 3. The apparatus according to claim 1 wherein the attribute represents an 

2" author of the input, and further wherein the human reviewer is selected based on a 

3 predetermined designation by the author of the input. 

f ^ 4. The apparatus according to claim 1 further comprising means for 

M transmitting the input and the output to the selected human reviewer for review. 

1 5. The apparatus according to claim 1 further comprising means for 

2 automatically identifying potential errors contained in the output. 

1 6. The apparatus according to claim 1 wherein the information processing 

2 system includes a speech-to-text system. 

1 7., An apparatus for facilitating review of an automatically transcribed 

2 document generated by a media conversion system, wherein the document is 

3 generated in response to an input, the apparatus comprising: 

4 a. means for extracting a keyword from the document; 

5 b. means for selecting one of a plurality of reviewers in response to the 

6 keyword; and 
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7 c. means for transmitting the input and the document to the selected 

8 reviewer for review. 

1 8. The apparatus according to claim 7 wherein the apparatus is 

2 configured for coupling to a speech-to-text system. 

1 9. The apparatus according to claim 7 further comprising means for 

2 storing a plurality of reviewer profiles, wherein each of the reviewer profiles 

3 corresponds to one of the plurality of reviewers, and further wherein the reviewer 

4 profiles are updated after the reviewer finishes review of the document. 

t;;f 10. The apparatus according to claim 9 wherein the keyword represents a 

|;i subject matter of the document and further wherein each of the reviewer profiles 

3 3 includes words commonly used within a specific field of knowledge. 



V a 1 1 - The apparatus according to claim 1 0 wherein the keyword represents 

2L a name of an author of the input. 



jh 12. The apparatus according to claim 10 wherein the means for selecting 

Z i compares the keyword with each of the reviewer profiles to select the reviewer 

33 according to a predetermined selection criteria. 

1 13. The apparatus according to claim 7 further comprising means for 

2 storing a plurality of user profiles, wherein each of the user profiles corresponds to 

3 one of a plurality of users of the apparatus, and further wherein the user profiles 

4 are adapted dynamically with corrections made by the reviewer. 

1 14. The apparatus according to claim 7 further comprising means for 

2 updating a list of available reviewers. 

1 1 5. The apparatus according to claim 7 further comprising: 

2 a. means for analyzing associate portions of the document with a 

3 plurality of confidence levels of conversion accuracy; 

- 12 - 
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4 b. means for marking the portions according to a corresponding one of 

5 the plurality of confidence levels; and 

6 c. means for displaying each portion in a predetermined color 

7 representing the corresponding one of the plurality of the confidence 

8 levels. 

1 1 6. The apparatus according to claim 1 5 wherein the means for analyzing 

2 is disposed to adaptively process the document according to information stored in 

3 the user profiles. 

1_ 1 7. The apparatus according to claim 7 further comprising: 

2§ a. means for selecting a portion of the document; and 

$|] b. means for outputting a portion of the input corresponding to the 

4^ portion of the document. 



18. An apparatus for improving productivity of human review of an 

2 automatically transcribed document generated by a speech-to-text conversion 

§2 system, wherein the document is generated from an audio recording recorded by an 

fk author, the apparatus comprising: 

|J a. a controller coupled to the speech-to-text conversion system for 

p£ selecting an appropriate one of a plurality of human reviewers to 

7 review the document wherein, the controller is capable of extracting 

8 an attribute from the document representative of a content of the 

9 document and further wherein the controller stores a plurality of 

10 reviewer profiles corresponding to each one of the human reviewers 

1 1 and provides the selected reviewer in response to the attribute; 

12 b. a transmission device coupled to the controller for transmitting the 

1 3 document and the audio recording to the selected reviewer; 

14 c. means for analyzing the transcription to associate portions of the 
1 5 transcription with a plurality of confidence levels of transcription 

16 accuracy, wherein each portion is marked according to one of the 

1 7 plurality of confidence levels; 



- 13 - 



PATENT 
OCTEL-00700 



1 8 d. means for selecting one of the portions of the document in response 

19 to a command from the reviewer; and 

20 e. means for playing a portion of the voice recording corresponding to a 

21 selected portion of the transcription. 

1 1 9. The apparatus according to claim 1 8 wherein the controller also stores 

2 a list of available reviewers, and further wherein the controller selects the selected 

3 reviewer in response to comparing the attribute with the reviewer profiles and the 

4 list of available reviewers. 

fif 20. The apparatus according to claim 18 wherein the attribute comprises a 

i;i list of keywords representative of a subject matter of the document and further 

3D wherein each of the reviewer profiles comprises words relevant to the subject 

4ft matter. 

f h 21 . The apparatus according to claim 1 8 wherein the attribute comprises a 

Zj name of the author of the voice recording and further wherein each of the reviewer 

$ 5 profiles comprises a list of authors that a corresponding reviewer has served. 

p 22. The apparatus according to claim 1 8 further comprising: 

~2* a. means for establishing a connection between an author of the voice 

3 recording and the reviewer; and 

4 b. displaying the document to solicit comments from the author. 

1 23. The apparatus according to claim 18 wherein the means for analyzing 

2 is disposed to adaptively process the document according to information stored in 

3 the user profiles. 

1 24. The apparatus according to claim 18 further comprising a means for 

2 displaying for displaying the document for the reviewer. 

1 25. The apparatus according to claim 24 wherein the means for displaying 

2 comprises a color monitor. 

- 14- 
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1 26. The apparatus according to claim 18 wherein the means for selecting 

2 comprises a speech recognition system for receiving verbal commands from the 

3 reviewer. 

1 27. The apparatus according to claim 1 8 wherein each portion of the 

2 document is displayed in a color representing a corresponding confidence level. 

1 28. The apparatus according to claim 18 wherein the means for playing 

2 further comprises means for varying a playback speed of the voice recording. 

|;f 29. A method of improving productivity of human review of an 

2S1 automatically transcribed document generated by an information processing 

3f system, wherein the document is generated in response to an input, the method 

4ii comprising the steps of: 

a. extracting an attribute from the document; and 
g !% b. selecting one of a plurality of human reviewers based on the attribute. 

|;": 30. The method according to claim 29 wherein the step of selecting 

15 includes comparing the attribute with a plurality of reviewer profiles each 

jp corresponding to a respective one of the plurality of reviewers. 

1 31 . The method according to claim 29 wherein the step of selecting 

2 includes the steps of: 

3 a. identifying an author of the input; 

4 b. retrieving stored data about the author, wherein the stored data 

5 includes a roster of the reviewers who have previously served the 

6 author; and 

7 c. choosing one of the reviewers from the roster. 
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1 32. The method according to claim 29 further comprising the steps of: 

2 a. automatically identifying a portion of the document which is potentially 

3 erroneous; 

4 b. playing an audio portion of the voice recording corresponding to the 

5 portion of the document upon a command of the human reviewer; and 

6 c. making corrections to the document. 

1 33. The method according to claim 32 wherein the audio portion of the 

2 voice recording is played back at a reviewer selectable speed. 

|;j 34. The method according to claim 29 further comprising the steps of: 

2;f a. analyzing the transcription with a plurality of confidence levels of 

3;i transcription accuracy; and 

4J b. displaying each of the portions in a predetermined color according to 

|s the confidence level. 

J,, 35. The method according to claim 29 further comprising the step of 

2;i automatically correcting grammatical and typographical errors of the transcription. 

p 36. The method according to claim 29 further comprising the steps of: 

§3 a. establishing a connection between an author of the voice recording 

3 and the reviewer; and 

4 b. displaying the transcription to the author to solicit comments. 
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ABSTRACT 



An apparatus for improving productivity of human reviewers of transcribed 
documents generated by media conversion systems includes a server/client network 
of computers, memories and file systems. The server receives and stores voice 
files created by users of the system. The server is configured for coupling to a 
speech-to-text media conversion system to receive converted text files of the audio 
voice files. The server analyzes the converted text files and routes the converted 
files to the appropriate reviewers according to an adaptive algorithm. The 
converted files are displayed on the assigned reviewer's screen at the reviewer's 
workstation. To aid the reviewer in pinpointing potential errors, the workstation 
displays different segments of the converted files in different colors to reflect 
different confidence levels of transcription accuracy. Portions of the original voice 
message that correspond to the potential errors are played back for the reviewer. 
The reviewers' workstations also perform productivity enhancing functions such as 
spelling and grammar checking. After the reviewer has made all the necessary 
corrections, the reviewed files are transmitted back to the server to be stored and 
accessed by the users. A user database in the server is also updated to store 
recurrent user-specific errors corrected by the reviewer. A language analysis 
system is also disposed to adaptively correct user-specific errors in future reviews 
according to the information in the user database. 
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IN THE UNITED STATES 
PATENT AND TRADEMARK OFFICE 

Declaration and Power of Attorney 
As the below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to 
my name. 

I believe I am the original, first and sole inventor of the subject matter which is 
claimed and for which a patent is sought on the invention entitled METHOD OF AND 
APPARATUS FOR IMPROVING PRODUCTIVITY OF HUMAN REVIEWERS OF 
AUTOMATICALLY TRANSCRIBED DOCUMENTS GENERATED BY MEDIA 
CONVERSION SYSTEMS the specification of which is attached hereto. 

I hereby state that I have reviewed and understand the contents of the above 
identified specification, including the claims, as amended by an amendment, if any, 
specifically referred to in this oath or declaration. 

I acknowledge the duty to disclose all information known to me which is 
material to patentability as defined in Title 37, Code of Federal Regulations, 1.56. 

I hereby claim foreign priority benefits under Title 35, United States Code, 119 
of any foreign application(s) for patent or inventor's certificate listed below and have 
also identified below any foreign application for patent or inventor's certificate having 
a filing date before that of the application on which priority is claimed: 

None 

I hereby claim the benefit under Title 35, United States Code, 120 of any 
United States application(s) listed below and, insofar as the subject matter of each of 
the claims of this application is not disclosed in the prior United States application in 
the manner provided by the first paragraph of Title 35, United States Code, 112, I 
acknowledge the duty to disclose all information known to me to be material to 
patentability as defined in Title 37, Code of Federal Regulations, 1 .56 which became 
available between the filing date of the prior application and the national or PCT 
international filing date of this application: 

None 

I hereby declare that all statements made herein of my own knowledge are 
true and that all statements made on information and belief are believed to be true; 
and further that these statements were made with the knowledge that willful false 
statements and the like so made are punishable by fine or imprisonment, or both, 
under Section 1001 of Title 18 of the United States Code and that such willful false 
statements may jeopardize the validity of the application or any patent issued 
thereon. 
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I hereby appoint the following attorney(s) with full power of substitution and 
revocation, to prosecute said application, to make alterations and amendments 

therein, to receive the patent, and to transact all business in the Patent and 
Trademark Office connected therewith: 

Thomas J. Bean (Reg. No. 44528) 

Lester H. Birnbaum (Reg. No. 25830) 

Richard J. Botos (Reg. No. 32016) 

Jeffery J. Brosemer (Reg. No. 36096) 

Kenneth M. Brown (Reg. No. 37590) 

Donald P. Dinella (Reg. No. 39961) 

Guy Eriksen (Reg. No. 41 736) 

Martin I. Finston (Reg. No. 31613) 

William S. Francos (Reg. No. 38456) 

Barry H. Freedman (Reg. No. 26166) 

Julio A. Garceran (Reg. No. 37138) 

Jimmy Goo (Reg. No. 36528) 

Anthony Grillo (Reg. No. 36535) 

Stephen M. Gurey (Reg. No. 27336) 

John M. Harman (Reg. No. 38173) 

Matthew J. Hodulik (Reg. No. 36164) 

Michael B. Johannesen (Reg. No. 35557) 

Mark A. Kurisko (Reg. No. 38944) 

Irena Lager (Reg. No. 39260) 

John B. Maclntyre (Reg. No. 41 170) 

Christopher N. Malvone (Reg. No. 34866) 

Scott W. McLellan (Reg. No. 30776) 

Martin G. Meder (Reg. No. 34674) 

John C. Moran (Reg. No. 30782) 

Michael A. Morra (Reg. No. 28975) 

Gregory J. Murgia (Reg. No. 41209) 

Claude R. Narcisse (Reg. No. 38979) 

Joseph J. Opalach (Reg. No. 36229) 

Neil R. Ormos (Reg. No. 35309) 

Eugen E. Pacher (Reg. No. 29964) 

Jack R. Penrod (Reg. No. 31864) 

Gregory C. Ranieri (Reg. No. 29695) 

Scott J. Rittman (Reg. No. 39010) 

Ferdinand M. Romano (Reg. No. 32752) 

Eugene J. Rosenthal (Reg. No. 36658) 

Bruce S. Schneider (Reg. No. 27949) 

Ronald D. Slusky (Reg. No. 26585) 

David L. Smith (Reg. No. 30592) 

Ozer M. N. Teitelbaum (Reg. No. 36698) 

John P. Veschi (Reg. No. 39058) 
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David Volejnicek 
Charles L. Warren 
Jeffrey M. Weinick 
Eli Weiss 



(Reg No. 29355) 
(Reg. No. 27407) 
(Reg. No 36304) 
(Reg. No. 17765) 



I hereby appoint the attorney(s) on ATTACHMENT A as associate attorney(s) 
in the aforementioned application, with full power solely to prosecute said 
application, to make alterations and amendments therein, to receive the patent, and 
to transact all business in the Patent and Trademark Office connected with the 
prosecution of said application. No other powers are granted to such associate 
attorney(s) and such associate attorney(s) are specifically denied any power of 
substitution or revocation. 



Full name of sole inventor; Kamil Grajski 



Inventor's 

signatur e ^LoaaaaA AA , /uiIaU^ Date f/tflv^ 



Residence: 5292 Atennines Circle, San Jose, CA 95138 
Citizenship: United States 

Post Office Address: 5292 Atennines Circle, San Jose, CA 95138 
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ATTACHMENT A 



Attorney Name(s): Thomas B. Haverstock Reg. No.: 32,571 



Jonathan O. Owens 37,902 



Telephone calls should be made to HAVERSTOCK & OWENS LLP at: 
Phone No.: (650) 833-0160 

Fax No.: (650) 833-01700 



All written communications are to be addressed to: 



HAVERSTOCK & OWENS LLP 
260 Sheridan Avenue, Suite 420 
Palo Alto, CA 94306 
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